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Abstract: Introduction: The Centre for Addiction and Mental Health (CAMH) is a 500 bed freestanding psychiatric 
hospital in Canada. We are in the process of preparing for an integrated commercial clinical information system, which 
will have computerized physician order entry (CPOE) functionality. 

Methods: As a preparation for CPOE, we developed inpatient order sets (OSs). Development teams from individual 
clinical programs created and sent their OSs to an OS Working Group for initial endorsement, and then to Pharmacy & 
Therapeutics and Medical Advisory committees subsequent approvals. 

Results: In twelve months we created and introduced 22 behavioral health OSs across eight clinical programs in our 
hybrid information system with an excellent adoption rate (>97%) by clinicians. 

Discussion: The development and implementation temporarily contributed to a multifactorial flow problem in the 
emergency department (ED), which was addressed by substantially simplifying the General Admission via the ED OS. 
Also, as the OSs were developed and sent for approval the project identified areas where local clinical practice can 
improve. Our electronic-paper hybrid set of clinical systems was a major factor impacting the effort. 
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INTRODUCTION 

The Centre for Addiction and Mental Health (CAMH) is 
a 500-bed freestanding mental health hospital with 3000 
employees, including 130 full-time medical staff. There are a 
total of eight Clinical Programs or service lines, and an 
active Primary Care service. We currently have a hybrid of 
electronic and paper documentation systems; physician 
orders are on paper with the exception of laboratory and 
dietary orders, which are electronic. Our order set (OS) 
initiative was launched to aid our preparations for a 
Computerized Provider Order Entry (CPOE) project that was 
imminent. To assist with this project a vendor was used to 
support the development of order sets (OSs) and the 
corresponding clinical process redesign work that are 
mandatory prerequisites to CPOE. The goal was to develop 
and implement OSs in a hybrid clinical information 
environment across eight inpatient programs over one year. 
Reasons for OSs are many [1], including 1) OSs are required 
as clinical content in preparation for Computerized Provider 
Order Entry (CPOE) [2,3] ; 2) Paper-based OSs will ready 
our staff for CPOE and standardization of care [4] ; and 3) 
OSs can improve quality [5-8], safety [9], resource 
utilization [10], order completeness, workflow, staff and 
patient satisfaction [11]. As is the case with the use of 
electronic systems more generally, OSs are widely used in 
physical medicine, but less so in behavioral health. As 
inpatient psychiatry gradually adopts electronic medical 
records, OSs will play a growing role in bringing clinical 
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content to electronic ordering. Our main research questions 
were: How readily will clinicians adopt paper order sets in a 
hybrid information system? What obstacles will need to be 
overcome? 

METHODS 

We applied a variant of the methodology recommended 
by our vendor, PatientOrderSets.com. In February an OS 
working group (OSWG) was formed and it consisted of 
representatives from physicians, nursing informatics, clinical 
forms administration, medical informatics, health 
information systems analyst, allied health, nursing (nurse 
educator and advanced practice nurse), health records, 
laboratory, and clinical informatics. The OSWG began 
having weekly meetings in March 2010. Its mandate 
included: 

1) support the creation and maintenance of standardized 
modular OSs; 

2) establish a Standard Reference OS with the oversight 
of PatientOrderSets.com for a guideline in the 
creation of CAMH OSs; 

3) facilitate multidisciplinary communication and 
collaboration; 

4) ensure compliance with Institute and Safe Medication 
Practices (ISMP) standards, hospital abbreviation 
standards, hospital formulary, and any hospital 
policies that may influence OS content. 

Meetings were set with all physicians of each clinical 
program and with managers and their clinical staff to inform 
them of the implementation plan, educate them about OSs 
and convey what was expected of them. 
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The eight clinical inpatient programs and primary care 
were informed that members from each program were 
required to participate on their local program content teams 
in April. There was an initial meeting with each content team 
in May to inform them of the time line (6 weeks) to develop 
their OSs complete with input and local consensus from all 
physicians in their program [12]. Each team would meet 
weekly there after to develop content for their respective 
programs; they were also responsible to seek feedback and 
agreement from all clinicians, including review by Clinical 
Program Medical Directors. Throughout the development 
phase, the OSWG would also meet weekly to review and 
recommend changes to the originating content team. 

After final revisions were completed on the OSs, they 
were submitted for approval. The first level of approval was 
from the OSWG; OSs then went on to the Pharmacy and 
Therapeutic Committee for subsequent approval; then lastly 
to the Medical Advisory Committee (MAC) for final vetting. 
Table 1 shows all the OSs that were finally approved by the 
MAC. 

The paper OSs consisted of physician orders, alerts and 
clinical information. After final approval at MAC, OSs were 
implemented. Communications were sent to all clinicians 



outlining instructions, implementation dates, available 
education, support and contact information. The medical 
staff received training in the use of OS at their usually 
scheduled physician meetings and were supplied all relevant 
implementation information. Throughout the implementation 
of the emergency department all residents on their ED 
rotation were educated in the use of the paper OSs. Clinical 
staff provided post-implementation comments and 
suggestions to the lead of the OSWG. 

Implementation was over three months starting in late 
September and finishing mid-December. 

During the implementation of the twenty-two OSs, 
clinical support was provided to each area in the form of a 
clinical informatics nurse physically present for 12 hours 
daily, and then being available via phone for the remaining 
12 hrs. Initial support lasted one to three weeks depending 
on the size of the program. Following the initial support 
period, 24-hour phone support by the clinical informatics 
nurse went for a period of 6 months (Table 2). 

This project was approved by our institutional research 
ethics board. In order to help assure support and engagement 
at all levels of the organization there was regular 



Table 1. Order Sets with Corresponding Content Teams 



Order Set Number 


Order Set 


Content Team 


1 


ED/EAU*** Alcohol Withdrawal with Diazepam 


CATS* 


2 


Mood and Anxiety Disorders Direct Admission 


Mood and Anxiety 


3 


Bowel Care 


Primary Care 


4 


Diabetes Management 


Primary Care 


5 


Electroconvulsive Therapy (ECT****) Referral 


Mood and Anxiety 


6 


Electroconvulsive Therapy (ECT****) Treatment 


Mood and Anxiety 


7 


Electroconvulsive Therapy (ECT****) Direct Admission 


Geriatric Mental Health 


8 


ED/EAU Hypertension Crisis 


Primary Care 


9 


Dual Diagnosis Direct Admission 


Dual Diagnosis 


10 


Dual Diagnosis Privileges and Passes 


Dual Diagnosis 


11 


General Admission via the ED 


CATS* 


12 


General Psychiatry Unit/Acute Care Unit Direct Admission 


CATS* 


13 


Law and Mental Health Direct Admission 


LAMHP** 


14 


Medical Withdrawal Service + 21 Day Program Direct Admission 


Addictions 


15 


Metabolic Monitoring 


Schizophrenia 


16 


Women's Program Direct Admission 


Women's Program 


17 


Geriatric Mental Health Dysphagia Management 


Geriatric Mental Health 


18 


Geriatric Mental Health Falls Prevention 


Geriatric Mental Health 


19 


Geriatric Mental Health Behavioral Management 


Geriatric Mental Health 


20 


Geriatric Mental Health Direct Admission 


Geriatric Mental Health 


21 


Schizophrenia Direct Admission 


Schizophrenia 


22 


Nicotine Replacement Therapy (NRT) 


Primary Care/Addictions 



* Centralized Assessment, Triage and Support. 

** Law and Mental Health Program. 

*** Emergency Department/ Extended Assessment Unit. 

**** Electroconvulsive Therapy. 
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communication with the executive team which started in 
January 2010. Communications and updates to all clinicians 
were ongoing for the project's duration. 

Table 2. Order Sets Implementation Schedule 



Clinical Program 


Date 


Geriatric Mental Health 


September 27 


Addictions 


October 4 


Schizophrenia 


October 12 


Law and Mental Health (LAMHP) 


October 24 


CATS 


November 8 


Women's Program 


November 22 


Mood and Anxiety 


December 6 


Dual Diagnosis 


December 13 



RESULTS 

The clinical programs employed OSs and the direct 
admission OSs were added to each unit's routine paper 
admission packet. Auditing of admission OS use was done 
six months post-implementation and we observed that 
150/152 (98.7%) of all admissions in March and April 201 1 
used a Direct Admission or General Admission via the ED 
OS. Sixty percent of audited admissions used the General 
Admission via the Emergency Department (ED) OS, as the 
ED is the entry route for most CAMH inpatients. Nursing 
signatures were complete in 146/150 (97.3%) of admissions. 
Post-implementation reports from nursing indicated that 
nurses were very much in favour of the paper-based OSs for 
their completeness, clarity, and efficiency in providing care. 
At the same time, reports from the Emergency Department 
(ED) informed the OSWG that their admission OS was too 
lengthy. 

DISCUSSION 

Throughout the development and implementation stages 
communication with and engagement of clinicians was 
essential to the success of the project. Post-implementation 
reports from nursing indicated that nurses were very much in 
favour of the paper-based OSs for their completeness, 
clarity, and efficiency in providing care. 

Several issues were encountered during implementation 
and were responded to with iterative modifications to order 
sets. Changes requested by clinical areas were made, revised 
OSs were approved, put into clinical use, and that cycle was 
repeated as necessary. Nursing highlighted one key area: 
nurses questioned why they were required to sign off items 
on an OS that were not doctors' orders. Sign off of such 
items was required in some OSs because some of the content 
teams had tried to use the OSs to improve care quality by 
including nursing tasks that do not require a physician order, 
or by including certain kinds of documentation (e.g. alerts). 
The OSs have since been reviewed and unnecessary orders, 
fields and items have been removed. A second issue was the 
implementation of paper OSs in a hybrid paper-electronic set 
of clinical systems. Hybrid OSs were needed, with the 
majority being on paper, while lab and dietary orders were 
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electronic. Specific groups of lab orders were created, one 
for each OS, within our electronic ordering system. The 
splitting of OSs into digital and analog portions made 
necessary by our hybrid environment reduced the usability of 
OSs. An accompanying benefit was a rise in the rate of 
electronic lab orders at admission. Thirdly, most inpatient 
areas introduced OSs and a separate and new inpatient 
medication reconciliation procedure at the same time, so 
staff had to accommodate two process changes. 

As noted above, another problem requiring resolution 
happened in the Emergency Department (ED). Our ED is a 
high-volume area staffed mainly by a large number of 
residents who rotate, with any given resident providing 
periodic coverage. Medical staff and residents in the ED 
found that the admission OS was too long - combined with 
the new admission medication reconciliation steps, ED 
admission ordering became too complex. Both medication 
reconciliation and OS projects had an opportunity to simplify 
their processes: the CATS (ED) content team reduced the 
General Admission via the ED OS from three pages to one. 
The revised abbreviated version was accepted by medical 
staff and residents in the ED and is currently in use. 

There were some limitations to the OS design and 
implementation. Usability testing [13, 14] was not part of the 
project, and would have helped to avoid the flow problem in 
the ED. For speed and ease of implementation with limited 
resources, we focused on the pragmatic local consensus 
method [12], based on our standard of care. Lastly, the 
foundation of all CAMH OSs was a standard reference OS 
based on the vendor's standard reference OS derived from 
acute care settings; as such the standard reference OS was 
not behavioral health-specific. 

CONCLUSIONS 

Over one year, we established an approvals process, 
developed OS content, and implemented twenty-two 
inpatient OSs in eight clinical programs. To our knowledge, 
this is the first report on order sets from a freestanding 
psychiatric facility with a hybrid information environment. 
Reflecting our hybrid clinical systems, the OSs themselves 
were hybrid, principally paper-based, with corresponding 
electronic dietary and laboratory portions for each OS. 
Initially the OSs were designed to be quite comprehensive; 
experience of the hybrid OSs by ED and nursing staff has 
since led to review, with scrutiny to augment efficiency. 
Despite a hybrid information environment, and a separate 
concurrent medication reconciliation project, clinical staff 
fully embraced OSs for inpatient behavioral health. In the 
future, we will increase evidence-based content, convert the 
OSs to fully electronic versions for CPOE, and add mental 
health protocols and pathways. 
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